Network For Image Segmentation Research Articles

ConvMedSegNet:A multi-receptive field depthwise convolutional neural network for medical image segmentation

In order to achieve highly precise medical image segmentation, this paper presents ConvMedSegNet, a novel convolutional neural network designed with a U-shaped architecture that seamlessly integrates two crucial modules: the multi-receptive field depthwise convolution module (MRDC) and the guided fusion module (GF). The MRDC module’s primary function is to capture texture information of varying sizes through multi-scale convolutional layers. This information is subsequently utilized to enhance the correlation of global feature data by expanding the network’s width. This strategy adeptly preserves the inherent inductive biases of convolution while concurrently amplifying the network’s ability to establish dependencies on global information. Conversely, the GF module assumes responsibility for implementing multi-scale feature fusion by connecting the encoder and decoder components. It facilitates the transfer of information between features that are separated over substantial distances through guided fusion, effectively minimizing the loss of critical data. In experiments conducted on public medical image datasets such as BUSI and ISIC2018, ConvMedSegNet outperforms several advanced competing methods, yielding superior results. Additionally, the code can be accessed at https://github.com/csust-yixin/ConvMedSegNet.

Multimodal parallel attention network for medical image segmentation

Medical image segmentation is a crucial aspect of medical image processing, and has been widely used in the detection and clinical diagnosis for brain, lung, liver, heart and other diseases. In this paper, we propose a novel multimodal parallel attention network, called MPA-Net, for medical image segmentation. MPA-Net is divided into two parts. The first part extracts more high-dimensional features by improved network structure, which contains the skip connection, the output of the multimodal parallel attention and the output of the previous upsampling layer. The second part incorporates a multimodal parallel attention mechanism, encompassing feature parallel attention, spatial parallel attention and channel parallel attention. This mechanism facilitates the effective fusion of high-dimensional and low-dimensional features, leading to enhanced context information. Experimental results on Kagglelung dataset, Liver dataset, Cell dataset, Drive dataset and Kvasir-SEG dataset show that MPA-Net has achieved better segmentation performance than that of other baseline methods, on lung, liver, cell contour, retinal vessel and polyps.

Bilateral Supervision Network for Semi-Supervised Medical Image Segmentation.

Massive high-quality annotated data is required by fully-supervised learning, which is difficult to obtain for image segmentation since the pixel-level annotation is expensive, especially for medical image segmentation tasks that need domain knowledge. As an alternative solution, semi-supervised learning (SSL) can effectively alleviate the dependence on the annotated samples by leveraging abundant unlabeled samples. Among the SSL methods, mean-teacher (MT) is the most popular one. However, in MT, teacher model's weights are completely determined by student model's weights, which will lead to the training bottleneck at the late training stages. Besides, only pixel-wise consistency is applied for unlabeled data, which ignores the category information and is susceptible to noise. In this paper, we propose a bilateral supervision network with bilateral exponential moving average (bilateral-EMA), named BSNet to overcome these issues. On the one hand, both the student and teacher models are trained on labeled data, and then their weights are updated with the bilateral-EMA, and thus the two models can learn from each other. On the other hand, pseudo labels are used to perform bilateral supervision for unlabeled data. Moreover, for enhancing the supervision, we adopt adversarial learning to enforce the network generate more reliable pseudo labels for unlabeled data. We conduct extensive experiments on three datasets to evaluate the proposed BSNet, and results show that BSNet can improve the semi-supervised segmentation performance by a large margin and surpass other state-of-the-art SSL methods.

Efficient skin lesion segmentation with boundary distillation.

Medical image segmentation models are commonly known for their complex structures, which often render them impractical for use on edge computing devices and compromising efficiency in the segmentation process. In light of this, the industry has proposed the adoption of knowledge distillation techniques. Nevertheless, the vast majority of existing knowledge distillation methods are focused on the classification tasks of skin diseases. Specifically, for the segmentation tasks of dermoscopy lesion images, these knowledge distillation methods fail to fully recognize the importance of features in the boundary regions of lesions within medical images, lacking boundary awareness for skin lesions. This paper introduces pioneering medical image knowledge distillation architecture. The aim of this method is to facilitate the efficient transfer of knowledge from existing complex medical image segmentation networks to a more simplified student network. Initially, a masked boundary feature (MBF) distillation module is designed. By applying random masking to the periphery of skin lesions, the MBF distillation module obliges the student network to reproduce the comprehensive features of the teacher network. This process, in turn, augments the representational capabilities of the student network. Building on the MBF distillation module, this paper employs a cascaded combination approach to integrate the MBF distillation module into a multi-head boundary feature (M2BF) distillation module, further strengthening the student network's feature learning ability and enhancing the overall image segmentation performance of the distillation model. This method has been experimentally validated on the public datasets ISIC-2016 and PH2, with results showing significant performance improvements in the student network. Our findings highlight the practical utility of the lightweight network distilled using our approach, particularly in scenarios demanding high operational speed and minimal storage usage. This research offers promising prospects for practical applications in the realm of medical image segmentation.

MSGAT: Multi-scale gated axial reverse attention transformer network for medical image segmentation

Medical image segmentation can help doctors accurately identify and locate various structures, tissues, or lesions in the patient's body, providing an important basis for clinical diagnosis. However, the segmentation of medical images is seriously affected by the complexity of the structures in the images, the variety of morphologies, the uneven contrast, and the blurring of the boundaries between the target tissues and the background. Therefore, we propose a Multiscale Gated Axial Reverse Attention Transformer (MSGAT) for medical image segmentation. First, we use the Pyramid Vision Transformer (PVT) as the backbone network to extract the high-level features and introduce the Channel Received Field Block (CRFB) to filter the interfering information in the high-level features, and then we augment and aggregate the high-level features separately. Specifically, on the one hand, the Multiscale Feature Enhancement Module (MSFEM) is used to obtain multiscale contextual information to enhance the positional information of the target, and on the other hand, Multiscale Parallel Partial Decoder (MSPPD) is employed to aggregate the high-level feature maps to generate rough segmentation maps. Finally, the rough segmentation map is used to guide the gated axial reverse attention (GARA) combined with the position information in the enhanced feature map to refine the segmentation boundaries and gradually mine the detailed information of the target. MSGAT is evaluated against state-of-the-art methods on skin cancer (ISIC2016, PH2), polyp (Kvair), and gland (GlaS) datasets. Results demonstrate MSGAT's superiority, with an average improvement of 6.19% and 9.35% in mIoU and DSC metrics, respectively, compared to existing methods.

Artificial intelligence-enhanced automation for M-mode echocardiographic analysis: ensuring fully automated, reliable, and reproducible measurements.

To enhance M-mode echocardiography's utility for measuring cardiac structures, we developed and evaluated an artificial intelligence (AI)-based automated analysis system for M-mode images through the aorta and left atrium [M-mode (Ao-LA)], and through the left ventricle [M-mode (LV)]. Our system, integrating two deep neural networks (DNN) for view classification and image segmentation, alongside an auto-measurement algorithm, was developed using 5,958M-mode images [3,258M-mode (LA-Ao), and 2,700M-mode (LV)] drawn from a nationwide echocardiographic dataset collated from five tertiary hospitals. The performance of view classification and segmentation DNNs were evaluated on 594M-mode images, while automatic measurement accuracy was tested on separate internal test set with 100M-mode images as well as external test set with 280 images (140 sinus rhythm and 140 atrial fibrillation). Performance evaluation showed the view classification DNN's overall accuracy of 99.8% and segmentation DNN's Dice similarity coefficient of 94.3%. Within the internal test set, all automated measurements, including LA, Ao, and LV wall and cavity, resonated strongly with expert evaluations, exhibiting Pearson's correlation coefficients (PCCs) of 0.81-0.99. This performance persisted in the external test set for both sinus rhythm (PCC, 0.84-0.98) and atrial fibrillation (PCC, 0.70-0.97). Notably, automatic measurements, consistently offering multi-cardiac cycle readings, showcased a stronger correlation with the averaged multi-cycle manual measurements than with those of a single representative cycle. Our AI-based system for automatic M-mode echocardiographic analysis demonstrated excellent accuracy, reproducibility, and speed. This automated approach has the potential to improve efficiency and reduce variability in clinical practice.

A decoder-free feature aggregation network for medical image segmentation

A decoder-free feature aggregation network for medical image segmentation

CCFNet: Collaborative Cross-Fusion Network for Medical Image Segmentation

The Transformer architecture has gained widespread acceptance in image segmentation. However, it sacrifices local feature details and necessitates extensive data for training, posing challenges to its integration into computer-aided medical image segmentation. To address the above challenges, we introduce CCFNet, a collaborative cross-fusion network, which continuously fuses a CNN and Transformer interactively to exploit context dependencies. In particular, when integrating CNN features into Transformer, the correlations between local and global tokens are adaptively fused through collaborative self-attention fusion to minimize the semantic disparity between these two types of features. When integrating Transformer features into the CNN, it uses the spatial feature injector to reduce the spatial information gap between features due to the asymmetry of the extracted features. In addition, CCFNet implements the parallel operation of Transformer and the CNN and independently encodes hierarchical global and local representations when effectively aggregating different features, which can preserve global representations and local features. The experimental findings from two public medical image segmentation datasets reveal that our approach exhibits competitive performance in comparison to current state-of-the-art methods.

Learning A Single Network for Robust Medical Image Segmentation with Noisy Labels.

Robust segmenting with noisy labels is an important problem in medical imaging due to the difficulty of acquiring high-quality annotations. Despite the enormous success of recent developments, these developments still require multiple networks to construct their frameworks and focus on limited application scenarios, which leads to inflexibility in practical applications. They also do not explicitly consider the coarse boundary label problem, which results in sub-optimal results. To overcome these challenges, we propose a novel Simultaneous Edge Alignment and Memory-Assisted Learning (SEAMAL) framework for noisy-label robust segmentation. It achieves single-network robust learning, which is applicable for both 2D and 3D segmentation, in both Set-HQ-knowable and Set-HQ-agnostic scenarios. Specifically, to achieve single-model noise robustness, we design a Memory-assisted Selection and Correction module (MSC) that utilizes predictive history consistency from the Prediction Memory Bank to distinguish between reliable and non-reliable labels pixel-wisely, and that updates the reliable ones at the superpixel level. To overcome the coarse boundary label problem, which is common in practice, and to better utilize shape-relevant information at the boundary, we propose an Edge Detection Branch (EDB) that explicitly learns the boundary via an edge detection layer with only slight additional computational cost, and we improve the sharpness and precision of the boundary with a thinning loss. Extensive experiments verify that SEAMAL outperforms previous works significantly.

RCEAU-Net: Cascade Multi-Scale Convolution and Attention-Mechanism-Based Network for Laser Beam Target Image Segmentation with Complex Background in Coal Mine.

Accurate and reliable pose estimation of boom-type roadheaders is the key to the forming quality of the tunneling face in coal mines, which is of great importance to improve tunneling efficiency and ensure the safety of coal mine production. The multi-laser-beam target-based visual localization method is an effective way to realize accurate and reliable pose estimation of a roadheader body. However, the complex background interference in coal mines brings great challenges to the stable and accurate segmentation and extraction of laser beam features, which has become the main problem faced by the long-distance visual positioning method of underground equipment. In this paper, a semantic segmentation network for underground laser beams in coal mines, RCEAU-Net, is proposed based on U-Net. The network introduces residual connections in the convolution of the encoder and decoder parts, which effectively fuses the underlying feature information and improves the gradient circulation performance of the network. At the same time, by introducing cascade multi-scale convolution in the skipping connection section, which compensates for the lack of contextual semantic information in U-Net and improves the segmentation effect of the network model on tiny laser beams at long distance. Finally, the introduction of an efficient multi-scale attention module with cross-spatial learning in the encoder enhances the feature extraction capability of the network. Furthermore, the laser beam target dataset (LBTD) is constructed based on laser beam target images collected from several coal mines, and the proposed RCEAU-Net model is then tested and verified. The experimental results show that, compared with the original U-Net, RCEAU-Net can ensure the real-time performance of laser beam segmentation while increasing the Accuracy by 0.19%, Precision by 2.53%, Recall by 22.01%, and Intersection and Union Ratio by 8.48%, which can meet the requirements of multi-laser-beam feature segmentation and extraction under complex backgrounds in coal mines, so as to further ensure the accuracy and stability of long-distance visual positioning for boom-type roadheaders and ensure the safe production in the working face.

SAHIS-Net: a spectral attention and feature enhancement network for microscopic hyperspectral cholangiocarcinoma image segmentation

SAHIS-Net: a spectral attention and feature enhancement network for microscopic hyperspectral cholangiocarcinoma image segmentation

Hi-gMISnet: generalized medical image segmentation using DWT based multilayer fusion and dual mode attention into high resolutionpGAN.

Automatic medical image segmentation is crucial for accurately isolating target tissue areas in the image from background tissues, facilitating precise diagnoses and procedures. While the proliferation of publicly available clinical datasets led to the development of deep learning-based medical image segmentation methods, a generalized, accurate, robust, and reliable approach across diverse imaging modalities remains elusive. This paper proposes a novel high-resolution parallel generative adversarial network (pGAN)-based generalized deep learning method for automatic segmentation of medical images from diverse imaging modalities. The proposed method showcases better performance and generalizability by incorporating novel components such as partial hybrid transfer learning, discrete wavelet transform (DWT)-based multilayer and multiresolution feature fusion in the encoder, and a dual mode attention gate in the decoder of the multi-resolution U-Net-based GAN. With multi-objective adversarial training loss functions including a unique reciprocal loss for enforcing cooperative learning in pGANs, it further enhances the robustness and accuracy of the segmentation map. Experimental evaluations conducted on nine diverse publicly available medical image segmentation datasets, including PhysioNet ICH, BUSI, CVC-ClinicDB, MoNuSeg, GLAS, ISIC-2018, DRIVE, Montgomery, and PROMISE12, demonstrate the proposed method's superior performance. The proposed method achieves mean F1 scores of 79.53%, 88.68%, 82.50%, 93.25%, 90.40%, 94.19%, 81.65%, 98.48%, and 90.79%, respectively, on the above datasets, surpass state-of-the-art segmentation methods. Furthermore, our proposed method demonstrates robust multi-domain segmentation capabilities, exhibiting consistent and reliable performance. The assessment of the model's proficiency in accurately identifying small details indicates that the high-resolution generalized medical image segmentation network (Hi-gMISnet) is more precise in segmenting even when the target area is very small. The proposed method provides robust and reliable segmentation performance on medical images, and thus it has the potential to be used in a clinical setting for the diagnosis of patients.

Self-supervised multi-magnification feature enhancement for segmentation of hepatocellular carcinoma region in pathological images

Accurate segmentation of hepatocellular carcinoma regions is a critical step in surgical evaluation of whole slide pathological images. Recently, emerging multi-magnification learning-based methods have shown promise with evaluation of whole slide images. However, traditional multi-magnification learning segmentation models focus on uniformity and fail to effectively exploit different magnification information. The purpose of this study is to develop a novel multi-magnification learning segmentation model by effectively utilizing diverse magnification information. Therefore, we developed a novel multi-magnification feature self-enhancement network to extract magnification-specific features from multi-magnification images via self-supervised learning without extra supervision. Specifically, the proposed network enhances feature information via a super-resolution module based on self-supervision to transfer the representative information into a segmentation encoder, which takes full advantage of the pyramid storage characteristic of whole slide images. Moreover, a multi-scale feature fusion module based on the attention mechanism is used to fuse pretrained multi-scale features through a gated unit block, which is designed to fuse features from generation task into and segmentation task. We evaluated this method using the hepatocellular carcinoma data set from The Cancer Genome Atlas with cross-validation and achieved a 0.829 Dice similarity coefficient, nearly a 3% improvement over state-of-the-art models used for segmentation. To the best of our knowledge, this is the first study of the application of super-resolution as a self-enhancement network for histopathological image segmentation via self-supervised pretext learning. The code will be available at https://github.com/SH-Diao123/Self-Supervised-Multi-magnification-Segmentation.

Superpixel-Based Graph Convolutional Network for UAV Forest Fire Image Segmentation

Given the escalating frequency and severity of global forest fires, it is imperative to develop advanced detection and segmentation technologies to mitigate their impact. To address the challenges of these technologies, the development of deep learning-based forest fire surveillance has significantly accelerated. Nevertheless, the integration of graph convolutional networks (GCNs) in forest fire detection remains relatively underexplored. In this context, we introduce a novel superpixel-based graph convolutional network (SCGCN) for forest fire image segmentation. Our proposed method utilizes superpixels to transform images into a graph structure, thereby reinterpreting the image segmentation challenge as a node classification task. Additionally, we transition the spatial graph convolution operation to a GraphSAGE graph convolution mechanism, mitigating the class imbalance issue and enhancing the network’s versatility. We incorporate an innovative loss function to contend with the inconsistencies in pixel dimensions within superpixel clusters. The efficacy of our technique is validated on two different forest fire datasets, demonstrating superior performance compared to four alternative segmentation methodologies.

STCS-Net: a medical image segmentation network that fully utilizes multi-scale information

In recent years, significant progress has been made in the field of medical image segmentation through the application of deep learning and neural networks. Numerous studies have focused on optimizing encoders to extract more comprehensive key information. However, the importance of decoders in directly influencing the final output of images cannot be overstated. The ability of decoders to effectively leverage diverse information and further refine crucial details is of paramount importance. This paper proposes a medical image segmentation architecture named STCS-Net. The designed decoder in STCS-Net facilitates multi-scale filtering and correction of information from the encoder, thereby enhancing the accuracy of extracting vital features. Additionally, an information enhancement module is introduced in skip connections to highlight essential features and improve the inter-layer information interaction capabilities. Comprehensive evaluations on the ISIC2016, ISIC2018, and Lung datasets validate the superiority of STCS-Net across different scenarios. Experimental results demonstrate the outstanding performance of STCS-Net on all three datasets. Comparative experiments highlight the advantages of our proposed network in terms of accuracy and parameter efficiency. Ablation studies confirm the effectiveness of the introduced decoder and skip connection module. This research introduces a novel approach to the field of medical image segmentation, providing new perspectives and solutions for future developments in medical image processing and analysis.

UCTNet: Uncertainty-guided CNN-Transformer hybrid networks for medical image segmentation

Transformer, born for long-range dependency establishment, has been widely studied as a complementary of convolutional neural networks (CNNs) in medical image segmentation. However, existing CNN-Transformer hybrid approaches simply pursue implicit feature fusion without considering their underlying functional overlap. Medical images typically follow stable anatomical structures, making convolution capable of handling most segmentation targets. Without differentiation, enforcing transformers to operate self-attention for all image patches would result in severe redundancy, hindering global feature extraction. In this paper, we propose a simple yet effective hybrid network named UCTNet where transformers only focus on establishing global dependency for CNN’s unreliable regions predicted through uncertainty estimation. In this way, CNN and transformer are explicitly fused to minimize functional overlap. More importantly, with fewer regions to handle, UCTNet is of better convergence to learn more robust feature representations for hard examples. Extensive experiments on publicly-available datasets demonstrate the superiority of UCTNet against the state-of-the-art approaches, achieving 89.44%, 92.91%, and 91.15% in Dice similarity coefficient on Synapse, ACDC, and ISIC2018 respectively. Furthermore, such a CNN-Transformer hybrid strategy is highly extendable to other frameworks without introducing additional computational burdens. Code is available at https://github.com/innocence0206/UCTNet.

SASAN: Spectrum-Axial Spatial Approach Networks for Medical Image Segmentation.

Ophthalmic diseases such as central serous chorioretinopathy (CSC) significantly impair the vision of millions of people globally. Precise segmentation of choroid and macular edema is critical for diagnosing and treating these conditions. However, existing 3D medical image segmentation methods often fall short due to the heterogeneous nature and blurry features of these conditions, compounded by medical image clarity issues and noise interference arising from equipment and environmental limitations. To address these challenges, we propose the Spectrum Analysis Synergy Axial-Spatial Network (SASAN), an approach that innovatively integrates spectrum features using the Fast Fourier Transform (FFT). SASAN incorporates two key modules: the Frequency Integrated Neural Enhancer (FINE), which mitigates noise interference, and the Axial-Spatial Elementum Multiplier (ASEM), which enhances feature extraction. Additionally, we introduce the Self-Adaptive Multi-Aspect Loss (LSM), which balances image regions, distribution, and boundaries, adaptively updating weights during training. We compiled and meticulously annotated the Choroid and Macular Edema OCT Mega Dataset (CMED-18k), currently the world's largest dataset of its kind. Comparative analysis against 13 baselines shows our method surpasses these benchmarks, achieving the highest Dice scores and lowest HD95 in the CMED and OIMHS datasets. Our code is publicly available at https://github.com/IMOP-lab/SASAN-Pytorch.

PAMSNet: A medical image segmentation network based on spatial pyramid and attention mechanism

The image segmentation of diseases can help clinical diagnosis and treatment in medical image analysis. Due to the complexity of lesion features (e.g., size, location, and morphology) and the high similarity between the background and the target area in medical images, semantic features are difficult to extract completely. To tackle these problems, we propose a novel medical image segmentation network PAMSNet based on the spatial pyramid and attention mechanism. By using efficient pyramid attention and channel spatial attention modules, the proposed method fuses the extracted multi-scale spatial information with the local features extracted by the encoder to supplement the image details. In addition, the Spatial Pyramid-Coordinate Attention (SPCA) module is introduced in the bottleneck layer to obtain larger receptive field information and enhance feature extraction. We conducted qualitative and quantitative evaluations on four public datasets, including ISIC2018, Lung segmentation, Kvasir-SEG, and ISLES2022. The segmentation accuracy of DSC was 87.86%, 98.18%, 82.43%, and 87.37%, respectively. The ablation study of each part of PAMSNet proves the validity of each component, and the comparison with state-of-the-art methods on different indicators proves the predominance of the network.

DS-MSFF-Net: Dual-path self-attention multi-scale feature fusion network for CT image segmentation

DS-MSFF-Net: Dual-path self-attention multi-scale feature fusion network for CT image segmentation

PIS-Net: Efficient Medical Image Segmentation Network with Multivariate Downsampling for Point-of-Care.

Recently, with more portable diagnostic devices being moved to people anywhere, point-of-care (PoC) imaging has become more convenient and more popular than the traditional "bed imaging". Instant image segmentation, as an important technology of computer vision, is receiving more and more attention in PoC diagnosis. However, the image distortion caused by image preprocessing and the low resolution of medical images extracted by PoC devices are urgent problems that need to be solved. Moreover, more efficient feature representation is necessary in the design of instant image segmentation. In this paper, a new feature representation considering the relationships among local features with minimal parameters and a lower computational complexity is proposed. Since a feature window sliding along a diagonal can capture more pluralistic features, a Diagonal-Axial Multi-Layer Perceptron is designed to obtain the global correlation among local features for a more comprehensive feature representation. Additionally, a new multi-scale feature fusion is proposed to integrate nonlinear features with linear ones to obtain a more precise feature representation. Richer features are figured out. In order to improve the generalization of the models, a dynamic residual spatial pyramid pooling based on various receptive fields is constructed according to different sizes of images, which alleviates the influence of image distortion. The experimental results show that the proposed strategy has better performance on instant image segmentation. Notably, it yields an average improvement of 1.31% in Dice than existing strategies on the BUSI, ISIC2018 and MoNuSeg datasets.